Teaching Quality and Student Outcomes: Academic Achievement and Educational Engagement in Rural Northwest China

A central task of educational researchers has been to uncover factors that improve student academic achievement. Research in both developed and developing nations during the past few decades has analysed the links between educational outcomes and school physical resources, teacher quality and children's demographic and family background. Importantly, research on teacher and school effects in developing countries has focused on factors such as human capital, economic resources and physical infrastructure, the so-called input factors in the "black box" production function model of school outcomes. Fewer studies have focused on the "softer" classroom process factors that might be seen as important mechanisms of the production function, such as teaching style, the quality of teacher-student interactions and student academic engagement. Disciplines International and Comparative Education Comments Reprinted with permission. Reprinted in China: An International Journal, Volume 5, Issue 2, September 2007, pages 309-334. Publisher URL: http://muse.jhu.edu/journals/china/ This journal article is available at ScholarlyCommons: http://repository.upenn.edu/gansu_papers/5 © CHINA: AN INTERNATIONAL JOURNAL 5, 2 (SEP. 2007): 309 – 334 ABOUT THE AUTHORS Xuehui An ([email protected]) is Associate Professor at the National Center for Education Development Research in China’s Ministry of Education. She did her PhD in the economics of education at Beijing Normal University. Her research interests include teacher incentives and school management in rural areas, education finance and education policy. Emily Hannum ([email protected]) is Assistant Professor at the University of Pennsylvania. She did her PhD in sociology at the University of Michigan. Her research interests include Chinese society, sociology of education, social stratification, poverty and child welfare and global development. Tanja Sargent ([email protected]) is Assistant Professor at Rutgers, The State University of New Jersey. She did her PhD in education and sociology at the University of Pennsylvania. Her research interests include Chinese education reform, sociology of education, schools as organisations and basic education in developing countries. COMMENTS AND NOTES Teaching Quality and Student Outcomes: Academic Achievement and Educational Engagement in Rural Northwest China Xuehui AN, Emily HANNUM and Tanja SARGENT A central task of educational researchers has been to uncover factors that improve student academic achievement. Research in both developed and developing nations during the past few decades has analysed the links between educational outcomes and school physical resources, teacher quality and children’s demographic and family background.1 Importantly, research on teacher and school effects in developing countries has focused on factors such as human capital, economic resources and physical infrastructure, 07 C&N An p309-334.indd 309 8/16/07 5:18:16 PM 310 XUEHUI AN, EMILY HANNUM AND TANJA SARGENT 1 For a review, see Claudia Buchmann and Emily Hannum, “Education and Stratification in Developing Countries: A Review of Theories and Research”, Annual Review of Sociology, no. 27 (2001): 77–102. 2 For reviews, see Buchmann and Hannum, “Education and Stratification in Developing Countries”; Bruce Fuller, “What School Factors Raise Achievement in the Third World?”, Review of Educational Research 57, no. 3 (1987): 255–92; and Bruce Fuller and Prema Clarke, “Raising School Effects While Ignoring Culture? Local Conditions and the Influence of Classroom Tools, Rules, and Pedagogy”, Review of Educational Research 64, no. 1 (1994): 119–57. the so-called input factors in the “black box” production function model of school outcomes. Fewer studies have focused on the “softer” classroom process factors that might be seen as important mechanisms of the production function, such as teaching style, the quality of teacher-student interactions and student academic engagement. This study investigates the sensitivity of academic achievement and educational engagement to student experiences in the classroom: teaching style, teacher-student interactions and classroom environment. This study is first placed in the theoretical context of comparative educational research, then in the context of recent education reform initiatives in China. Next follows a description of the data which came from a survey of primary school students, teachers and principals in rural Northwest China that was carried out in the summer of 2000. Multivariate analyses of achievement and engagement are presented and the paper closes with a discussion of the implications of the findings. Framework and Hypotheses Research on Teacher Quality Disparate lines of research in the fields of sociology, demography and economics have considered community, school, classroom and family characteristics that predict children’s educational outcomes.2 Across these disciplines, an important goal has been to produce policy-relevant insights on measurable aspects of school quality, including teacher quality, that significantly affect outcomes such as achievement. Much of this research stemmed from Stephen Heyneman’s “Coleman Report for a Developing Country”, and his subsequent research suggesting that the portion of the variance in achievement attributable to school quality, as opposed to family background, was generally much larger 07 C&N An p309-334.indd 310 8/16/07 5:18:16 PM 311 ACADEMIC ACHIEVEMENT AND EDUCATIONAL ENGAGEMENT 3 Stephen P. Heyneman, “Influence on Academic Achievement: A Comparison of Results from Uganda and More Industrialized Societies”, Sociology of Education 46 (1976); and later Stephen P. Heyneman and William A. Loxley, “The Effect of Primary School Quality on Academic Achievement across Twenty-Nine Highand Low-Income Countries”, American Journal of Sociology 88 (1983). 4 Fuller, “What School Factors Raise Achievement in the Third World?”; and Fuller and Clarke, “Raising School Effects While Ignoring Culture?”. 5 D.P. Baker, B. Goesling and G.K. LeTendre, “Socioeconomic Status, School Quality and National Economic Development: A Cross-National Analysis of the ‘HeynemanLoxley Effect’ on Mathematics and Science Achievement”, Comparative Education Review 46, no. 3, (2002): 291–312; and D.P. Baker and G.K. LeTendre, National Differences, Global Similarities: World Culture and the Future of Schooling (Palo Alto: Stanford University Press, 2005). 6 Gary Burtless, “Introduction and Summary”, in Does Money Matter?: The Effect of School Resources on Student Achievement and Adult Success, ed. Gary Burtless (Washington, DC: Brookings Institution Press, 1996), pp. 1–42; Eric Hanushek, “Interpreting Recent Research on Schooling in Developing Countries”, World Bank Research Observer 10 (1995): 247–54; and Michael Kremer, “Research on Schooling: What We Know and What We Don’t (a comment on Hanushek)”, World Bank Research Observer 10 (1995): 247–54. 7 Eric Hanushek, “The Economics of Schooling: Production and Efficiency in Public Schools”, Journal of Economic Literature 24 (1986): 1141–77; Eric Hanushek, “The Impact of Differential Expenditures on School Performance”, Educational Researcher 1 (1989): 45–51; and Eric Hanushek, “School Resources and Student Performance”, in Does Money Matter?: The Effect of School Resources on Student Achievement and Adult Success, ed. Gary Burtless (Washington, DC: Brookings Institution Press, 1996), pp. 43–73. in developing versus industrialised countries.3 Following Heyneman’s lead, studies of school effects have been conducted in a wide range of developing countries, and many have found significant effects of school factors, net of family background, on achievement.4 However, recent work by David Baker and his colleagues has suggested that the Coleman effect — large family effects and small school effects — has spread throughout the world.5 Moreover, studies that have sought to identify effects of specific dimensions of teacher quality and other school inputs on academic achievement have produced decidedly mixed results for both developed and developing countries.6 For example, Hanushek has argued that US-based research yields no systematic evidence that teacher education, experience or salaries affect student performance.7 07 C&N An p309-334.indd 311 8/16/07 5:18:16 PM 312 XUEHUI AN, EMILY HANNUM AND TANJA SARGENT 8 Eric Hanushek, John F. Kain and Steven G. Rivkin, “Teachers, Schools and Academic Achievement”, National Bureau of Economic Research Working Paper, no. 6691, 1998; and David Card and Alan Krueger, “Labor Market Effects of School Quality: Theory and Evidence”, in Does Money Matter?: The Effect of School Resources on Student Achievement and Adult Success, ed. Gary Burtless (Washington, DC: Brookings Institution Press, 1996). 9 Heyneman and Loxley, “The Effect of Primary School Quality on Academic Achievement across Twenty-Nine Highand Low-Income Countries”; Fuller and Clarke, “Raising School Effects While Ignoring Culture?”; Hanushek, “Interpreting Recent Research on Schooling in Developing Countries”; and Kremer, “Research on Schooling: What We Know and What We Don’t”. 10 Jean Anyon, “Social Class and School Knowledge”, Curriculum Inquiry 11, no. 1 (1981); Annette Lareau, “Social Class Differences in Family-School Relationships”, Sociology of Education 60 (1987); Kimberly Goyette and Gilberto Q. Conchas, “Family and NonFamily Roots of Social Capital Among Vietnamese and Mexican American Children”, in Research in Sociology of Education 13: Schooling and Social Capital in Diverse Cultures, ed. Bruce Fuller and Emily Hannum (Oxford: Elsevier Science, 2002), pp. 41–72; Other studies have found stronger evidence of positive school and teacher effects on learning and labour market outcomes.8 In developing countries, a number of studies have found that teacher education and experience, as well as basic material resources do affect achievement, but other work has presented a mixed verdict on teacher and school effects.9 Limited Definition of Quality One problem with these studies has been the narrow conceptualisation of teacher quality. Too often, teacher quality has been conceptualised simplistically, as easy-to-measure background factors thought to be linked to productivity such as teacher education or training, teacher salary or teacher experience. While identifying a mix of easy-to-measure and easy-to-manipulate inputs has a great deal of theoretical and policy appeal, the lack of consistent findings suggests the value of exploring alternative approaches. One potentially fruitful approach is to look for quality not in the background attributes of teachers, but rather in classroom (or school) environments that they have a hand in creating. For example, one interesting development in sociology of education has focused on understanding social and cultural environments within schools and classrooms, and particularly the role of hospitable and inhospitable environments for learning.10 Educational research on effective teaching styles also carries interesting potential directions for conceptualising teacher quality. For example, one 07 C&N An p309-334.indd 312 8/16/07 5:18:17 PM 313 ACADEMIC ACHIEVEMENT AND EDUCATIONAL ENGAGEMENT Toby L. Parcel and Mikaela J. Dufur, “Capital at Home and at School: Effects on Student Achievement”, Social Forces 79, no. 3 (2001): 881–912; and An, Xuehui, “Shisheng ketang hudong guocheng yu xuexiao xiaolu guanli (Teacher-student Interactions and School Effectiveness)”, Journal of Northwest Normal University 2 (2006). 11 See the 12 principles of effective teaching that Jere Brophy presents in his monograph on teaching. Jere Brophy, Teaching (Brussels/Geneva: International Academy of Education/International Bureau of Education, 1999). 12 Brophy, “Teaching”; Yi Chen and Wei Li, “Xiaoxue jiaoshi renge tezheng he xuesheng xueye chengji xiangguan yanjiu (Study on the Relationship between the Personality Characteristics of Primary School Teachers and Student Academic Achievement)”, Nanjing shida xuebao (shehui kexue ban) Journal of Nanjing Normal University (Social Science), no. 4 (2000); and Nel Noddings, The Challenge to Care in Schools (New York: Teachers College Press, 1992). 13 Fuller and Clarke, “Raising School Effects While Ignoring Culture?”. 14 See for example, ibid., and L.J. Saha, “Social Structure and Teacher Effects on Academic Achievement: A Comparative Analysis”, Comparative Education Review 27 (1983): 69–88; and Cynthia Lloyd, Barbara Mensch and W.H. Clark, “The Effects of Primary School Quality on School Dropout among Kenyan Girls and Boys”, Comparative Education Review 44 (2000): 113–47. important dimension of teaching style is the extent to which students are given the opportunity to participate actively in the classroom, as opposed to listening passively to lectures. According to Jere Brophy, classrooms of successful teachers “feature more time spent in interactive discourse and less time spent in solitary seatwork ... [or] extended lecture presentations”.11 Brophy also suggests that discourse is important for fostering higher order thinking skills, and that discourse should not be limited to rapidly-paced recitation that elicits short answers to miscellaneous questions. Other aspects of teaching style that may matter for student outcomes are teacher characteristics such as enthusiasm and the extent to which teachers embody an “ethic of caring”, such as friendliness and affection for students.12 Such behaviour on the part of the teacher provides an exemplary model that can exert a socialising influence on the students’ behaviour. It also establishes a safe environment in which students collaborate with each other and mistakes are accepted as a part of the natural learning process. Fuller and Clarke assert that there is a lack of studies of teacher effects in developing countries investigating the teaching behaviours or classroom environmental factors that promote favourable student outcomes.13 The few studies that do exist have found significant effects on achievement or attainment of classroom management, hours of instruction and classroom dynamics.14 07 C&N An p309-334.indd 313 8/16/07 5:18:17 PM 314 XUEHUI AN, EMILY HANNUM AND TANJA SARGENT Limited Definitions of Student Outcomes Just as teacher quality has been narrowly defined, so have student outcomes, primarily as academic achievement or attainment. It is likely that teaching styles and the classroom environment may be linked directly to another concept that has received little attention in research on teacher quality: students’ academic engagement. In US-based sociology of education research, engagement is coming to be recognised as an important research topic. For example, Johnson, Crosnoe and Elder characterise the educational experience of American middle school and high school students as “a multifaceted phenomenon that encompasses far more than academic achievement and degree attainment, which have been the primary foci of sociological research”.15 Other important aspects of the educational experience include daily participation in school and students’ feelings about school. Johnson et al. argue that these latter, and much less understood, aspects of the educational experience also have important consequences in children’s lives. A child’s degree of educational engagement provides obvious benefits for his or her school performance, as measured by other educational outcomes. For example, US-based research indicates that academically engaged students are less likely to drop out of high school or to engage in other problematic behaviours.16 However, engagement may also be a fundamental component of what Johnson et al. refer to as the “good student” role: a role that may set the stage for successful functioning in non-school settings later in life. In other words, engagement may engender or reflect traits such as ambition and 15 Monica Kirkpatrick Johnson, Robert Crosnoe and Glen H. Elder, Jr., “Students’ Attachment and Academic Engagement: The Role of Race and Ethnicity”, Sociology of Education 74, no. 4 (2001): 318. 16 For example, Anthony S. Bryk and Yeow Meng Thum, “The Effects of High School Organisation on Dropping Out: An Exploratory Investigation”, American Educational Research Journal 26 (1989): 353–83; George Farkas, Robert P. Grobe, Daniel Sheehan and Yuan Shuan, “Cultural Resources and School Success: Gender, Ethnicity and Poverty Groups within an Urban School District”, American Sociological Review 55 (1990): 127–42; Patricia Jenkins, “School Delinquency and School Commitment”, Sociology of Education 68 (1995): 221–39; Johnson, Crosnoe and Elder, Jr., “Students’ Attachment and Academic Engagement”; and Fred Newmann, Gary Wehlage and Susie Lamborn, “The Significance and Sources of Student Engagement”, in Student Engagement and Achievement in American Secondary Schools, ed. Fred Newmann (New York: Teachers College Press, 1992), pp. 11–39. 07 C&N An p309-334.indd 314 8/16/07 5:18:17 PM 315 ACADEMIC ACHIEVEMENT AND EDUCATIONAL ENGAGEMENT confidence that determine future life outcomes independently of academic success.17 Thus, engagement is potentially important as an influence on future socio-economic status, either instrumentally through its effect on academic achievement and persistence, or directly by fostering character traits that predict success in the labour market. In either case, the importance of such effects suggests that educational engagement itself is an outcome worthy of study. This paper argues that engagement with schooling may be particularly important to understand in settings such as rural Gansu, where education beyond the primary level has placed a serious economic burden on families and children’s attitudes toward schooling may play a role in household decisions about how long to support children in school. China Context Student participation in interactive classroom discourse, a caring relationship between teachers and students, and students’ engagement with schooling have been the focus of recent attention among policy makers in China. The “New Curriculum” reforms, which began initial implementation in 2001, seek to reduce the traditional emphasis on rote memorisation, drill, monotonous classroom environments and a heavy burden of homework assignments.18 17 Johnson, Crosnoe and Elder, Jr., “Students’ Attachment and Academic Engagement”. 18 PRC Ministry of Education, “Jichu jiaoyu kecheng gaige gangyao (Shixing)” [Framework for the Curriculum Reform of Basic Education (Trial Version)] (Beijing: Ministry of Education, 2001); Ministry of Education, “Mianshang ershiyi shiji jiaoyu zhenxing xingdong jihua” (Action Plan Towards the Revitalisation of Education for the 21st Century) (PRC Ministry of Education, 1998); Ministry of Education, Suzhi jiaoyu guannian: xuexi tiyao (The Concept of Quality Education: Key Points for Study) (Beijing, 2002); Tanja Sargent, “Ideologies of Educational Purpose for the 21st Century: Curriculum Policy and the Transformation of Teaching Practices in China” (Unpublished dissertation, University of Pennsylvania, PA, 2006); Tanja Sargent, “Ideologies of Educational Purpose for the 21st Century: The ‘New Curriculum’ Reform Policy in China” (Paper presented at the American Educational Research Association Annual Meeting, San Francisco, CA, Apr. 2006); Tanja Sargent, “Revolutionizing Ritual Interaction in the Classroom: Constructing the Chinese Renaissance of the 21st Century” (Paper presented at the Eastern Sociological Society Annual Meeting, Boston, MA, Mar. 2006); and Ou Shi and Liqun Liu, Kecheng gaige zhong de ruogan wenti (Problems in Curriculum Reform) (Guangzhou: Guangdong Educational Press, 2004). 07 C&N An p309-334.indd 315 8/16/07 5:18:18 PM 316 XUEHUI AN, EMILY HANNUM AND TANJA SARGENT These reforms emphasise instead the importance of a more studentcentred teaching style that stimulates students to become active subjects of their own learning, engaged in inquiry and discussion, and imbued with a love of learning, self-confidence, self-discipline and cooperativeness.19 The new reforms place great emphasis on the creation of a relaxed and encouraging classroom environment that meets the needs of children of all abilities and backgrounds. Complicating implementation of these changes is the continuing reality of competitive examinations as the primary basis for advancement in the system. Whether progressive teaching styles will be as effective as traditional styles in preparing students for high-stakes tests remains an open question. Further, especially in China’s poorer regions where school fees place a real burden on families, exams are not the only determining factor in children’s school progress. Whether or not students and their parents see schooling as being worth the hardship it places on the family probably depends on students’ engagement with schooling — their aspirations, academic confidence, their industriousness and their alienation. For this reason, and because engagement is thought to predict achievement, it is also important to understand the relationship between teaching quality and engagement measures. The analysis focuses on data that were collected the year prior to the initial implementation of China’s curricular reforms, thus cannot investigate questions about the reforms, per se. The analysis can, however, address a significant, related question: how children’s perceptions of the behaviours encouraged by the reforms relate to their performance and to their engagement with the schooling process. Hypotheses and Analytic Approach To address the theoretical and policy issues outlined in the preceding section, the paper presents models of achievement and engagement that, first, assess the effects of teacher background characteristics, including those commonly 19 Wang Xia and W. Liu, “Xin kecheng linian xia ketang jiaoxue zhong de shisheng hudong (Teacher-Student Interactions in New Curriculum Classroom Teaching)”, Jichu jiaoyu cankao (Basic Education Reference), no. 2 (2005); Wensen Yu, “Shisheng hudong, gongtong fazhan: Xin kecheng suo changdao de jiaoxueguan” (Teachers and Student Interacting and Developing Together: The Teaching Philosophy of the New Curriculum), Shifan jiaoyu (Teacher Education), no. 9 (2002). 07 C&N An p309-334.indd 316 8/16/07 5:18:18 PM 317 ACADEMIC ACHIEVEMENT AND EDUCATIONAL ENGAGEMENT considered as dimensions of teacher quality and additional demographic variables, then assess the effects of variables linked to teaching quality and classroom environment. The hypotheses presented in this paper are as follows: (a) Achievement and engagement are higher in classrooms where students experience a safe and stable environment, where they perceive that learning is more interactive and homework is not excessive, where they experience closer relationships to teachers, and where they perceive teachers to be fair. (b) Engagement has a direct positive relationship with student academic achievement. The analysis is composed of two sets of models: a set of regressions of student academic achievement and a set of regressions of educational engagement. Standard teacher “input” measures are considered. In addition, new measures of teaching style, classroom environment and teacher-student interactions are developed. The models also incorporate extensive controls for student family background and fixed effects for unmeasured differences across schools.